Analysing performance of first year engineering students

Many students in the engineering disciplines do not complete their higher education degree and drop out. This problem is serious, especially for first-year university students. In this paper, we analyse how students earn the credits required for their successful completion of the first study year. Using the example of a European technical university with traditional classroom-based education, we identify three groups of students: those who pass, those who earn only enough credits for staying in the program, and those who fail. Important patterns can be found at the end of the first semester. We present a simple algorithm that identifies students who may benefit from early additional support, which would increase their chances of progression to the second year and improve the retention improvement for the university. The results are evaluated in four consecutive academic years. The data from years 2013/14 and 2014/15 have been used to develop and verify the prediction model. In study years 2015/16 and 2016/17 the model has been applied to predict at-risk students, where the university tutors intervened and provided additional support and a significant improvement was achieved

Evaluating Weekly Predictions of At-Risk Students at The Open University: Results and Issues

Improving student retention rates is a critical task not only for traditional universities but particularly in distance learning courses, which are in recent years rapidly gaining in popularity. Early indications of potential student failure enable the tutor to provide the student with appropriate assistance, which might improve the student’s chances of passing the course. Collated results for a course cohort can also assist course teams to identify problem areas in the educational materials and make improvements for future course presentations. Recent work at the Open University (OU) has focused on improving student retention by predicting which students are at risk of failing. In this paper we present the models implemented at the OU, evaluate these models on a selected course and discuss the issues of creating the predictive models based on historical data, particularly mapping the content of the current presentation to the previous one. These models were initially tested on two courses and later extended to ten courses

Developing predictive models for early detection of at-risk students on distance learning modules

Not all students who fail or drop out would have done so if they had been offered help at the right time. This is particularly true on distance learning modules where there is no direct tutor/student contact, but where it has been shown that making contact at the right time can improve a student’s chances. This paper explores the latest work conducted at the Open University, one of Europe’s largest distance learning institutions, to identify when is the optimum time to make student interventions and to develop models to identify the at-risk students in this time frame. This work in progress is taking real time data and feeding it back to module teams as the module is running. Module teams will be indicating which of the predicted at-risk students have received an intervention, and the nature of the intervention

Investigating students' use of self-assessments in higher education using learning analytics

Background Formative assessments are vital for supporting learning and performance but are also considered to increase the workload of teachers. As self-assessments in higher education are increasingly facilitated via digital learning environments allowing to offer direct feedback and tracking students' digital learning behaviour these constraints might be reduced. Yet, learning analytics do not make sufficient use of data on assessments. Aims This exploratory case study uses learning analytics methods for investigating students' engagement with self-assessments and how this relates to performance in the final exam and self-reported self-testing strategies. Materials & Methods The research study has been conducted at a European university in a twelve-weeks course of a Bachelor's program in Economic and Business Education including nenroll = 159 participants. During the semester, students were offered nine self-assessments each including three to eight tasks plus one mid-term and one exam-preparation self-assessment including all prior self-assessments tasks. The self-assessment interaction data for each student included: the results of the last self-assessment attempt, the number of processed self-assessment tasks, and the time spent on the last self-assessment attempt, the total self-assessment attempts, and the first as well as last access of each self-assessment. Data analytics included unsupervised machine learning and process mining approaches. Results Findings indicate that students use the self-assessments predominantly before summative assessments. Two distinct clusters based on engagement with self-assessments could be identified and engagement was positively related to performance in the final exam. The findings from learning analytics data were also indicated by students' self-reported use of self-testing strategies. Discussion With the help of multiple data from self-reports, formal exams, and a learning analytics system, the findings provided multiple perspectives on the use of self-assessments and their relationships with course performance. These findings call for applying assessment analytics and related frameworks in learning analytics as well as providing learners with related adaptive feedback. Conclusion Future research might investigate different (self-report) variables for clustering, other student cohorts or self-assessment forms.Peer Reviewe

Modelling student online behaviour in a virtual learning environment

In recent years, distance education has enjoyed a major boom. Much work at The Open University (OU) has focused on improving retention rates in these modules by providing timely support to students who are at risk of failing the module. In this paper we explore methods for analysing student activity in online virtual learning environment (VLE) - General Unary Hypotheses Automaton (GUHA) and Markov chain-based analysis - and we explain how this analysis can be relevant for module tutors and other student support staff. We show that both methods are a valid approach to modelling student activities. An advantage of the Markov chain-based approach is in its graphical output and in the possibility to model time dependencies of the student activities. Drahomira Herrmannova,Lucie Vachova,Jakub Kuzilek,Zdenek Zdrahal,Annika Wolf

Modelling student online behaviour in a virtual learning environment

In recent years, distance education has enjoyed a major boom. Much work at The Open University (OU) has focused on improving retention rates in these modules by providing timely support to students who are at risk of failing the module. In this paper we explore methods for analysing student activity in online virtual learning environment (VLE) -- General Unary Hypotheses Automaton (GUHA) and Markov chain-based analysis -- and we explain how this analysis can be relevant for module tutors and other student support staff. We show that both methods are a valid approach to modelling student activities. An advantage of the Markov chain-based approach is in its graphical output and in the possibility to model time dependencies of the student activities.Comment: In Proceedings of the 2014 Workshop on Learning Analytics and Machine Learning at the 2014 International Conference on Learning Analytics and Knowledge (LAK 2014

First-Year Engineering Students’ Strategies for Taking Exams

Student drop-out is one of the most critical issues that higher educational institutions face nowadays. The problem is significant for first-year students. These freshmen are especially at risk of failing due to the transition from different educational settings at high school. Thanks to the massive boom of Information and Communication Technologies, universities have started to collect a vast amount of study- and student-related data. Teachers can use the collected information to support students at risk of failing their studies. At the Faculty of Mechanical Engineering, Czech Technical University in Prague, the situation is no different, and first-year students are a vulnerable group similar to other institutions. The most critical part of the first year is the first exam period. One of the essential skills the student needs to develop is planning for exams. The presented research aims to explore the exam-taking patterns of first-year students. Data of 361 first-year students have been analysed and used to construct “layered” Markov chain probabilistic graphs. The graphs have revealed interesting behavioural patterns within the groups of successful and unsuccessful students.Peer Reviewe

OU Analyse: analysing at-risk students at The Open University

The OU Analyse project aims at providing early prediction of ‘at-risk’ students based on their demographic data and their interaction with Virtual Learning Environment. Four predictive models have been constructed from legacy data using machine learning methods. In Spring 2014 the approach was piloted and evaluated on two introductory university courses with about 1500 and 3000 students, respectively. Since October 2014 the predictions have been extended to include 10+ courses of different level. The OU Analyse dashboard has been implemented, for presenting predictions and providing a course overview and a view of individual students

Student Drop-out Modelling Using Virtual Learning Environment Behaviour Data

With the rapid advancement of Virtual Learning Environments (VLE) in higher education, the amount of available student data grows. Universities collect the information about students, their demographics, their study results and their behaviour in the online environment. By applying modelling and predictive analysis methods it is possible to predict student outcome or detect bottlenecks in course design. Our work aims at statistical simulation of student behaviour in the VLE in order to identify behavioural patterns leading to drop-out or passive withdrawal i.e. the state when a student is not studying, but he has not actively withdrawn from studies. For that purpose, the method called Markov chain modelling has been used. Recorded student activities in VLE (VLE logs) has been used for constructing of probabilistic representation that students will perform some activity in the next week based on their activities in the current week. The result is an instance of the family of absorbing Markov chains, which can be analysed using the property called time to absorption. The preliminary results show that interesting patterns in student VLE behaviour can be uncovered, especially when combined with the information about submission of the first assessment. Our analysis has been performed using Open University Learning Analytics dataset (OULAD) and research notes are available online (https://bit.ly/2JrY5zv)